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The objectivity of forensic science decision making has received increased attention and scrutiny. However, 
there are only a few published studies experimentally addressing the potential for contextual bias. Because of 
the esteem of DNA evidence, it is important to study and assess the impact of subjectivity and bias on DNA 
mixture interpretation. The study reported here presents empirical data suggesting that DNA mixture 
interpretation is subjective. When 17 North American expert DNA examiners were asked for their 
interpretation of data from an adjudicated criminal case in that jurisdiction, they produced inconsistent 
interpretations. Furthermore, the majority of 'context free' experts disagreed with the laboratory's pre-trial 
conclusions, suggesting that the extraneous context of the criminal case may have influenced the 
interpretation of the DNA evidence, thereby showing a biasing effect of contextual information in DNA 
mixture interpretation. 

© 2011 Forensic Science Society. Published by Elsevier Ireland Ltd. All rights reserved. 


Seeking and interpreting information in a biased way so that it fits 
existing beliefs, expectation, hope, or motivation is a result of how we 
reason and is widespread [1]. The potential for such biases in forensic 
science disciplines has been suggested before [2,3], and has now been 
highlighted by the National Academy of Science (NAS) report on 
Strengthening Forensic Science in the United States: A Path Forward [4]. It 
directly discusses “the potential for bias and error in human observers” 
(p. 8), and states that “the extent to which practitioners in a particular 
forensic discipline rely on human interpretation that could be tainted by 

error, [or] the threat of bias_[is] significant” (p. 9). Indeed, empirical 

research supports the effects of bias in some forensic disciplines; for 
example, in fingerprinting, the same forensic experts may arrive at 
different conclusions when identical evidence is presented within 
different extraneous contexts (e.g., whether the detective believes the 
suspect is guilty, or the suspect confessed) [5-8]. 

However, in contrast to other forensic disciplines, DNA is regarded as 
the gold standard of forensic science [9]. DNA has been held as objective 
and immune to subjectivity and bias; “In the past several years, it has 
become commonplace in the courts, in the media, and in much of the 
technical literature, to contrast the scientific and objective evidence 
supplied by DNA profiling, with the experiential or subjective opinions 
given by traditional forensic experts” [9] (p. 97). Indeed, even the NAS 
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distinguishes between “forensic science disciplines [that] are laboratory 
based (e.g., nuclear and mitochondrial DNA analysis, toxicology and 
drug analysis)” [4] (p. 38), and other forensic disciplines that are “based 
on expert interpretation of observed patterns (e.g., fingerprints, writing 
samples, toolmarks, bite marks, and specimens such as hair)” [4] (p. 38). 

If correct, then DNA analyses should be consistent and not affected by 
domain irrelevant contextual circumstances. It seems, however, that at 
least in complex situations (such as with DNA mixtures) DNA does require 
and rely on human examiners making a variety of subjective judgements 
that are susceptible to bias. Indeed, in contrast to the view that DNA is 
objective, some have proposed that DNA analysis interpretations may be 
subjective and may even be influenced by a variety of factors [10,11 ]. 

However, such claims - both for the subjectivity or for the objectivity - 
of DNA analysis have rarely been examined and tested through empirical 
research. To investigate the subjectivity and biasability of mixture DNA 
analysis we observed and compared the conclusions on identical DNA 
evidence that was presented within and between different extraneous 
contextual information. To properly investigate this issue, it was critical 
to: 1. conduct the study with qualified DNA expert analysts who conduct 
real casework in accredited laboratories, and 2. that the examiners 
genuinely believed the contextual information, as contrived context 
within an experimental setup does not have the effect or impact as that of 
genuinely believed real context [8]. 

To achieve these goals we used mixture DNA analysis from a real 
adjudicated criminal case, using records obtained through the Georgia 
Freedom of Information Act. The case we chose provided us with 
analysis within extraneous context. We then took the same DNA 
evidence and presented it to 17 independent North American DNA 
expert analysts, but without the potentially biasing contextual case 
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information. First we compared the consistency in interpretation and 
conclusion within those 17 examiners to assess subjectivity in DNA 
analysis. Then we compared between them and those who examined 
the DNA mixture within the extraneous context of the criminal case to 
assess biasability in DNA analysis. The DNA evidence related to a gang 
rape case in which one of the assailants testified against the other 
suspects in return for a lesser sentence as part of his cooperation in a 
plea bargain deal. However, those identified through the plea bargain 
denied any involvement in the rape. 

The mixture DNA from the sexual assault was examined by experts in 
the real criminal case, and their analysis and conclusions were that the 
suspects that were identified by the cooperative assailant could not be 
excluded from being contributors to the mixture. The establishment of 
this corroborating fact was essential to the prosecution of the suspects 
who claimed innocence. Under the law of that state where this act 
occurred, the testimony of the admitted rapist would not be admitted 
without corroborating evidence. Therefore the DNA conclusions were 
critical to prosecution. If the suspects were excluded by DNA, or even if 
the DNA was “inconclusive”, the incriminating testimony of the admitted 
rapist would most likely not be allowed. As potentially biasing as this 
domain irrelevant context was, if DNA was totally objective it should not 
have affected their analysis. 

In this study we took the original materials used by the DNA 
examiners that concluded that the suspect cannot be excluded, and 
presented them to 17 other DNA examiners, 'context free.' These 17 DNA 
examiners were all expert DNA analysts who were working casework in 
an accredited governmental laboratory in North America. Fourteen 
were female and three were male; their mean age was 40.7 (SD = 5.86), 
and their mean years of experience conducting DNA analysis was 8.9 
(SD = 3.96). Two examiners had a BSc, 12 had a MSc (either in biology or 
forensic science), and 2 had PhDs (one participant did not provide 
information on their level of education). 

We asked the 17 independent DNA examiners to examine the DNA 
mixture along with DNA profiles of the victim and three suspects (Table 1 ) 
(one of the suspects, suspect 3, was the point of interest, as he was 
determined as ‘cannot be excluded’ by the DNA examiners who examined 
his DNA within the potentially biasing context). The evidence presented to 
them was comprised from the electropherograms (Figs. 1 and 2) available 
to the original examiners, and included the Vaginal Sperm Fraction 
(Profiler +) and Vaginal Sperm fraction (CoFiler). They were also provided 
with the relevant contextual information that was provided to the original 
examiners, such as the concentration of DNA in the sperm fraction extract, 
the DNA amplification conditions, and capillary electrophoresis injection 
times. Each of the 17 DNA examiners independently examined the 
evidence, and gave one of three conclusions for each of the suspects: 
‘cannot be excluded’, ‘excluded’, or ‘inconclusive.’ 

In regard to suspect three, the results obtained from the 17 
independent DNA examiners varied. One examiner concluded that the 
suspect ‘cannot be excluded’, 4 examiners concluded ‘inconclusive’, and 


Table 1 

Suspect 3 portion of the aiieie chart. 


Locus 

S3 

D3 

14, 17 

vWA 

17, 18 

FGA 

22, 24 

D8 

14, 15 

D21 

28, 28 

D18 

13, 18 

D5 

12, 13 

D13 

10, 14 

D7 

9, 10 

D3 

No data 

D16 

9,13 

THOl 

7,8 

TPOX 

9,9 

CSFIPO 

11,11 


12 examiners concluded ‘exclude.’ The results are revealing in two 
respects: First, the fact that the 17 DNA examiners were not consistent in 
their conclusions, by itself, suggests that there is an element of subjectivity 
in DNA interpretation. If it was totally objective, then all the examiners 
would have reached the same conclusion, especially since they all work in 
the same laboratory and follow the same interpretation guidelines. The 
observed inconsistencies within the 17 examiners who conducted their 
analysis on the identical evidence, ‘context free,’ demonstrated subjectiv¬ 
ity in DNA mixture analysis, which may reflect individual differences (e.g., 
training, experience, personality, and motivation). It is interesting that 
even using the ‘gold standard’ [9] DNA, different examiners reach 
conflicting conclusions based on identical evidentiary data. 

Second, comparing the data between examiners, those from the 
context free condition to those who were exposed to the extraneous 
context condition, it is possible that the domain irrelevant information 
may have biased their interpretation. The DNA analysts who concluded 
that the suspect cannot be excluded within the biasing context of the 
criminal case, are in sharp contrast to the vast majority of examiners who 
examined the same evidence without this biasing context. Only 1 (out of 
17) gave the same conclusion as the original analysts, 16 other examiners 
reached a different and conflicting conclusion (either ‘exclude’, 12 
examiners, or ‘inconclusive’, 4 examiners). Thus, the extraneous context 
appears to have influenced the interpretation of the DNA mixture, 
however, it is always hard to draw scientific conclusions when dealing 
with methodologies involving real casework. 

It must be emphasized, however, that these effects were observed for 
a DNA mixture analysis. Previous research in forensic identification 
suggests that contextual influences are most powerful when the evidence 
is ambiguous, complex, and a ‘hard call’ [8]. When the data is clear and 
decisions are simple, then the power of context is diminished. Gill has 
been quoted to say that “If you show 10 colleagues a mixture, you will 
probably end up with 10 different answers” [12]. The difficulties and 
challenges presented by complex DNA mixture have been the focus of 
several discussions [13-21], and are an important component of‘expert 
systems’ and statistical computing that try to more objectively 
deconvolute and interpret DNA mixtures [22,23]. 

The study reported here, the first experimental study exploring 
DNA interpretation, demonstrates that DNA mixture interpretation 
has subjective elements and may be susceptible to bias and other 
contextual influences. Minimizing such potential effects is important, 
and may include specific training on bias issues, as well as procedures 
and best practices especially designed to limit contextual influences 
(such as sequential unmasking [24]). 

This study also demonstrates that all types of DNA analysis should 
not be lumped together as the “gold standard.” It is true, that in 
contrast with many areas of forensic science [25], identity testing 
using DNA has progressed to the point of general acceptance when 
complete profiles are obtained from a single DNA contributor [26]. 
Consistent with this level of acceptance in the scientific community, 
the courts in the United States and elsewhere equate identity with 
DNA profiles that include complete allelic data from 13 or more of the 
standard short tandem repeat loci (STRs). However, in cases where 
low numbers of template molecules are amplified [27], or where 
complex mixtures are analyzed, subjective conclusions are made by 
analysts. This is evidenced by our experiment and the case we discuss, 
however, one cannot estimate its magnitude and impact without 
more empirical studies. 

The great degree of variability in laboratory methods regarding DNA 
mixtures has been the subject of concern in the DNA community, and the 
Scientific Working Group on DNA Analysis Methods (SWGDAM). It is also 
important to note that while some laboratories in North America still 
report qualitative results such as “cannot exclude” without quantitative 
measure, the 2010 SWGDAM guidelines state that “The laboratory must 
perform statistical analysis in support of any inclusion that is determined 
to be relevant in the context of a case, irrespective of the number of alleles 
detected and the quantitative value of the statistical analysis.” [28] 
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Fig. 1. Sperm fraction eiectropherogram from victim's vaginal swab, after amplification with CoFiler (ABl). This electropherogram was given to analysts for interpretation. Genetic 
loci are indicated in boxes above alleles. 


These guidelines however are not binding, and are not required 
for The American Society of Crime Laboratory Directors Laboratory 
Accreditation Board {ASCLD/lAB) accreditation. Outside of North 
America, the International Society for Forensic Genetics (ISFG) DNA 
commission recommendations on the interpretation of mixtures 
strongly supports the use of likelihood ratios [16], and this 
approach is beginning to gain ground in North America. 

It is also important to note that while this is the first published 
empirical study of potential DNA bias, Butler of the NIST laboratories 
has conducted extensive studies of mixture analysis over several 
years, wherein he supplies a large number of volunteer laboratories 


identical DNA mixture data and asks for their analysis. The results of 
these excellent studies have been presented at conferences and are 
available at the NIST webpages [29], but have never been published in 
a peer-reviewed journal. 

An interesting and perhaps the most critical point for this paper is 
that Butler's research findings show that inclusion statistics for the 
same profiles (using the same data) varied over 10 logs, that is from 1 
in 434,600 to 1.18x10^^, using the exact same electropherograms 
[29]. Therefore, although the use of statistics is paramount, it does not 
resolve the issue of subjectivity and potential bias, the topic of this 
study. 
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Fig. 2. Sperm fraction eiectropherogram from victim's vaginal swab, after amplification with Profiler Plus (ABl). This electropherogram was given to analysts for interpretation. 
Genetic loci are indicated in boxes above allele. 


The work presented here is a step in addressing the subjectivity and 
potential for bias in DNA mixture interpretation. It is clear that additional 
and follow-up studies are called for. However, acknowledging the role of 
the human examiner, understanding the role (and weaknesses) of human 
cognition in making forensic comparisons (including DNA mixtures), is an 
important step in correctly conceptualizing forensic science and finding 
ways for improvements [30]. 

Appendix A. Supplementary data 

Supplementary data to this article can be found online at doi:10. 
1016/j.scijus.2011.08.004. Additional information can be found at: 
www.cci-hq.com. 
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